Add CorruptedTsFileException for TsFile corruption errors with file path#18304
Add CorruptedTsFileException for TsFile corruption errors with file path#18304shuwenwei wants to merge 6 commits into
Conversation
…ath during query execution - Introduce CorruptedTsFileException extending RuntimeException with Stage enum (READ_TIMESERIES_METADATA, READ_CHUNK_DATA, LOAD_PAGE_READER, DECODE_PAGE_DATA, READ_METADATA_INDEX_NODE) - Uses super(message) + addSuppressed(cause) to prevent getRootCause penetration - Carries File reference and Stage for precise error reporting - Catch points in FileLoaderUtils, SeriesScanUtil, and DeviceCollector wrap IOException/RuntimeException as CorruptedTsFileException - Normal queries: message mentions corruption, tells user to check logs (no file path) - read_tsfile queries: message includes full TsFile path - DiskChunkLoader/DiskAlignedChunkLoader expose getTsFile() for File reference - ErrorHandlingUtils handles CorruptedTsFileException with TSFILE_PROCESSOR_ERROR - AbstractDriverThread matches CorruptedTsFileException by instanceof - AbstractDriverThread catch CorruptedTsFileException as abort cause - i18n: add EXCEPTION_ constants for all five stages (en + zh) - IT: IoTDBQueryWithCorruptedTsFileIT tests metadata index and page data corruption
CorruptedTsFileException is a RuntimeException and the DataNode handles it fine without wrapping, so the catch-and-rewrap block is redundant.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #18304 +/- ##
============================================
+ Coverage 43.02% 43.08% +0.05%
Complexity 374 374
============================================
Files 5362 5363 +1
Lines 381599 381773 +174
Branches 49553 49586 +33
============================================
+ Hits 164192 164474 +282
+ Misses 217407 217299 -108 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| * TsFileRuntimeException} so it follows the existing TsFile error handling and bypasses all {@code | ||
| * catch (IOException)} blocks in the operator hierarchy. | ||
| */ | ||
| public class CorruptedTsFileException extends TsFileRuntimeException { |
There was a problem hiding this comment.
extends IoTDBRuntimeException, add a new TSStatusCode for it, then add a else-if branch in ErrorHandlingUtils.onQueryException which won't print exception stack and only print message to info log
| // itself and we can match it here. | ||
| CorruptedTsFileException corruptedTsFileException = | ||
| (CorruptedTsFileException) rootCause; | ||
| logger.warn( |
There was a problem hiding this comment.
if it's external tsfile, info is enough, for internal tsfile, maybe warn is better.
|
JackieTien97
left a comment
JackieTien97
left a comment
There was a problem hiding this comment.
Additional findings after reviewing the latest revision.
| } | ||
|
|
||
| return timeSeriesMetadata; | ||
| } catch (Exception e) { |
There was a problem hiding this comment.
[P2] Please preserve non-TsFile failures before wrapping. This try also covers context.getPathModifications(...); its loader can throw MemoryNotEnoughException while reserving matched-mod memory. For a closed, healthy file, this catch converts QUERY_EXECUTION_MEMORY_NOT_ENOUGH into a corruption error and loses the original status. Narrow the catch to reader/deserialization operations or rethrow typed IoTDBRuntimeExceptions first. The aligned path below has the same issue.
| } | ||
| return chunkReader.loadPageReaderList(); | ||
| try { | ||
| return chunkReader.loadPageReaderList(); |
There was a problem hiding this comment.
[P2] For current disk readers this catch is unreachable: ChunkReader and AbstractAlignedChunkReader build page readers in constructors called by getChunkReader() above, while AbstractChunkReader.loadPageReaderList() only returns the prebuilt list. Page-header construction (and aligned time-page decompression) is therefore reported as READ_CHUNK_DATA, never LOAD_PAGE_READER. Please move the stage boundary to where construction occurs or remove/collapse this stage, and add an exact-stage test.
|
|
||
| // Corrupt the nodeType byte to 0xFF (all valid types are 0-3) | ||
| byte[] fileBytes = Files.readAllBytes(tsFile.toPath()); | ||
| fileBytes[(int) fileMetadataPos - 1] = (byte) 0xFF; |
There was a problem hiding this comment.
[P2] fileMetadataPos - 1 does not locate a device-index node in this fixture. With two devices, the device root is stored in TsFileMetadata; this byte belongs to the last measurement-index node. The passing IT consequently reports READ_TIMESERIES_METADATA, matching the broad assertion below, and never exercises READ_METADATA_INDEX_NODE/DeviceCollector. Force an on-disk device-index node or locate its offset explicitly, then assert the exact stage/message.
| int dataStart = magicLen + Byte.BYTES; | ||
| int dataEnd = (int) metaOffset; | ||
| // Corrupt bytes in the middle of the data section — XOR 512 bytes to ensure decompression fails | ||
| int middle = dataStart + (dataEnd - dataStart) / 2; |
There was a problem hiding this comment.
[P2] The midpoint is not a page-data boundary. In the current run both this and the normal-query case corrupt a chunk header and report READ_CHUNK_DATA, so DECODE_PAGE_DATA is untested and layout/compressor changes can alter the failure stage. Please locate a concrete page payload, fully consume the ResultSet, and assert the exact stage-specific message/log.
2 participants
Description
When a TsFile is corrupted during query execution, the original IOException from the TsFile reader is wrapped as a generic RuntimeException without the corrupted file path, making it impossible for users to locate the problem file.
This PR introduces
CorruptedTsFileExceptionto carry the corrupted TsFile path and the stage at which corruption was detected.Key design decisions:
catch (IOException)don't intercept itsuper(message) + addSuppressed(cause): preventsgetRootCause()from penetrating to the original IOException, allowingAbstractDriverThreadto match it byinstanceofFiles changed:
CorruptedTsFileException.java— exception with Stage enum and File referenceErrorHandlingUtils.java— handle CorruptedTsFileException with TSFILE_PROCESSOR_ERRORAbstractDriverThread.java— match CorruptedTsFileException as abort causeFileLoaderUtils.java— wrap IOException at READ_TIMESERIES_METADATA, READ_CHUNK_DATA, LOAD_PAGE_READER stagesSeriesScanUtil.java— wrap IOException at DECODE_PAGE_DATA stageDiskChunkLoader.java/DiskAlignedChunkLoader.java— expose getTsFile() for File referenceExternalTsFileQueryResource.java— wrap at READ_METADATA_INDEX_NODE stageDataNodeQueryMessages.java(en + zh) — EXCEPTION_ constants for all 5 stagesIoTDBQueryWithCorruptedTsFileIT.java— 3 test cases (metadata index + page data corruption for read_tsfile, page data for normal query)